Add rate limit HTTP service for external rate limits #1460
Conversation
Moves repo quotas from concurrent launches to a time-based rate limit
|
This also solves the same problem for a single binderhub with multiple replicas, where rate limits are per-replica instead of per deployment. An alternative to the custom HTTP endpoint is to use an external store like redis/etcd. It would add a dependency, but might be the better, more general approach. |
|
I feel like this gets into the bigger question of how to manage BinderHub's long term maintainability and scalability. We've done a bit of work on decoupling some components (e.g. running in K8s vs Docker).... do we split out more components and deal with their orchestration with Helm/terraform/etc, or build things into BinderHub since it's easier to deploy? |
|
I'm increasingly leaning toward directly supporting redis as the storage for rate limits. It would be a lot less code to maintain, and deploying a simple redis instance is not a big deal. I'd do that at the mybinder.org-deploy level, rather than binderhub, though. binderhub would only need to take redis connection info. |
Related to jupyterhub/mybinder.org-deploy#2143
The idea:
The rate limit implementation is the same (RateLimiter objects), but wrapped in an HTTP request when using the shared impelementation.
Notably, this weighs sessions that only last 10 seconds equally with sessions that last 3 hours. Currently active sessions does map onto 'cost' more directly, so maybe it is better to keep the concurrent session quota.
Still lots to do (tests and docs and such), but a sketch worth talking about